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Objective:  The  goal  of  our  contract  is  to  produce  and  deliver  technologies  for  coordinat¬ 
ing  the  behavior  of  large  numbers  of  form  factor  constrained  mobile  robots.  These  systems 
are  treated  as  massively  redundant  and  therefore  present  many  solutions  for  a  given  prob¬ 
lem.  The  project  contributes  technologies  that  extend  conventional  approaches  for  planning 
coordinated  behavior  and  for  programming  reactive  swarms.  The  approach  is  informed  by 
biology,  where  many  examples  of  societal  organizations  (specifically  in  insects)  exist,  and  by 
control  theory,  where  methods  exist  to  coordinate  and  guarantee  performance  of  redundant 
systems.  The  project  has  realized  both  simulated  and  real  experimental  work  and  has  demon¬ 
strated  coordination  constraints  expressed  in  a  control  theoretic  framework.  The  project  will 
demonstrate  how  a  distributed  implementation  with  localized  computation  and  communica¬ 
tion  can  guarantee  performance,  can  scale  to  hundreds  or  thousands  of  individuals,  and  how 
optimization  techniques  (like  reinforcement  learning)  can  be  used  in  this  context  to  acquire 
skillful,  distributed  control  policies. 


Approach:  The  UMass  approach  brings  traditions  in  control,  scheduling,  real-time  com¬ 
puting,  and  discrete  event  systems  together  with  dynamic  programming-based  reinforce¬ 
ment  learning.  The  UMass  project  advances  a  programming  and  analysis  methodology  for 
distributed  robots  with  which  to  build  Behavior  Programs  (B-Pgms)  for  coordinating  the 
activity  of  multiple  mobile  robots  with  stringent  form  factors.  Our  methodology  involves 
both  simulated  and  real  experimental  work.  We  wish  to  demonstrate  how  coordination  con¬ 
straints  can  be  expressed  in  a  control  theoretic  framework,  how  global  specifications  can 
be  guaranteed  in  a  distributed  implementation  with  localized  computation  and  communica¬ 
tion,  and  how  this  approach  scales  to  hundreds  of  individuals.  We  also  wish  to  demonstrate 
that  exploration-based  learning  algorithms  (like  reinforcement  learning)  can  be  used  in  this 
context  to  acquire  skillful,  distributed  control  policies.  The  control  framework  employs  a 
combination  of  closed-loop  controllers  that  actively  preserve  local  properties  in  the  system, 
a  method  for  combining  them  to  address  multiple,  concurrent  objectives  using  cascaded  null 
space  projections,  and  coordination  templates  for  combining  controllers  across  platforms 
that  actively  maintain  constraints  for  pairs  of  robots.  Pairwise  policies  have  been  used  to 
demonstrate  search-and-map,  attack-repel,  leader-follower,  and  network  connectivity  behav¬ 
ior  among  groups  of  independent  robots. 

The  off-line  programming  environment  uses  of  a  closed-loop  sensorimotor  instruction  set 
characterized  as  a  Markov  Decision  Problem  (MDP)  and  constrained  in  a  Discrete  Event 
framework  to  permit  only  correct  programs  in  the  distributed  system.  Correctness  includes 
notions  of  safety,  real-time  and  memory  constraints,  kinematic  performance  limitations,  and 
functional  specifications.  Both  user  and  automated  programming  techniques  are  employed 
to  address  currently  inadequate  levels  of  programmability  in  distributed,  sensor-based  ap¬ 
plications. 

A  run-time  environment  has  been  constructed  that  manages  the  execution  of  comprehen¬ 
sive,  distributed  software  on  modest  target  processors.  The  goal  is  to  accumulate  a  library 
of  correct  programs  cached  in  a  large  flash  memory  to  respond  to  a  variety  of  tasks  and 
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contingencies  using  code  that  cannot  reside  simultaneously  in  working  memory.  Behavior 
programs  are  replicated  in  memory  on  each  robot  to  coordinate  distributed  behavior  that 
is  coarsely  synchronized  by  low-bandwidth  interprocess  communications.  Global  correctness 
specifications  are  thus  preserved  in  the  compute-bound,  distributed  controller. 

SDR  Concept  and  Platform 

We  have  constructed  10  prototype  platforms  based  on  the  StrongARM  processor.  Examples 
of  the  control  methodology  have  been  developed  whereby  simple  skill  sets  in  single  robots 
are  acquired  and  used  to  express  peer-to-peer  coordination  strategies.  Finally,  team  oriented 
collaborations  have  been  constructed  from  these  peer-to-peer  policies. 

uBots  interact  in  the  context  of  an  information  framework  that  is  serviced  by  other  indepen¬ 
dent  teams  of  uBots  and  human  operators.  We  are  trying  to  make  optimal  control  decisions 
hierarchically  by  gathering  “intelligence”  in  the  form  of  additional  state  from  robot  teams 
with  narrow,  focused  objectives.  Teams  of  teams  are  configured  and  arranged  to  meet  control 
objectives  that  are  robust  with  respect  to  environmental  variation  and  respond  to  run-time 
feedback.  We  have  implemented  such  a  hardware  platform  in  the  form  of  multi-uBots  and 
auxiliary  sensors  that  are  distributed  spatially  and  networked.  The  sensor  network  consists 
of  panoramic  cameras,  pan/tilt/zoom  (PTZ)  cameras,  pyroelectric  sensors,  and  a  communi¬ 
cation  network  over  which  to  combine  features  across  multiple  sensors. 

Virtual  and  augmented  reality  interfaces  have  been  constructed  to  exchange  information 
between  humans  and  a  distributed  set  of  robots.  The  wearable  form  factor  enables  use  of 
the  interface  in  the  field,  where  the  user  becomes  a  participant  in  the  data  gathering  process. 
The  augmentation  of  the  external  world  enables  a  user  to  observe  otherwise  invisible  world 
states  (e.g.  chemical  hazards),  and  observe  areas  of  the  local  space  that  are  occluded  from 
the  user’s  current  point  of  view.  This  information  can  be  painted  on  top  of  the  external 
world  (augmenting  the  user’s  sensory  abilities),  or  can  virtually  bring  the  user  to  a  different 
location  from  which  to  observe  the  world. 

Our  project  aimes  to  understand  how  exploit  structure  for  learning  distributed  robot  be¬ 
havior.  We  have  developed  various  incarnations  of  an  interaction-based  state  estimator  to 
structure  the  representation  of  multi-objective  actions.  We  are  studying  learning  algorithms 
for  reconfigurable  stereo  vision  and  view  planning  for  localizing  and  tracking  human  subjects. 

Systems  and  Algorithms 

We  have  made  steady  progress  in  algorithm  and  system  developments  in  the  following  as¬ 
pects. 

Adaptive  Impedance  Control 

We  have  continued  to  develop  an  interaction-based  state  estimator  for  optimizing  tracking 
behavior  of  navigation  controllers.  A  new  representation  is  being  used  called  impedance 
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codes  (or  i-codes)  that  observes  tracking  performance  through  multiple  impedance  filters  and 
employs  one  of  them  to  actuate  the  motor  system.  We  have  constructed  switching  policies 
indexed  by  i-codes  to  optimize  performance  with  respect  to  time,  energy,  and  precision. 
Moreover,  we  are  now  experimenting  with  concurrent  combinations  of  time-,  energy-,  and 
precision-optimal  policies  that  automatically  encode  multiple  objectives  in  a  single  control 
expression. 

Multi-Robot  Coordination  and  Roles 

We  conducted  research  regarding  adaptive  role  switching  among  robots  to  increase  overall 
performance  of  the  team.  We  examined  the  task  of  exploring  a  set  of  rooms  while  maintaining 
line-of-sight  connectivity  among  a  team  of  robots.  We  have  developed  a  model  of  interaction- 
based  state  that  classifies  the  dynamics  of  the  active  controllers  of  the  constituent  robots. 
We  anticipated  that  learning  to  switch  roles  on  this  basis  would  increase  the  efficiency  of 
the  search  while  maintaining  the  line  of  sight  constraint.  However,  our  experiments  indicate 
that  there  is,  in  general,  little  advantage  in  this  task  to  switching  role. 

Real-Time  Process  Scheduling 

Intelligent  coordination  schemes  form  new  peer-to-peer  relationships  at  run-time  to  deal  op¬ 
portunistically  with  features  of  the  environment  that  are  discovered  during  execution.  To 
do  so,  we  have  begun  to  develop  a  scheduling  algorithm  that  can  verify  the  schedulability  of 
a  variety  of  expected  run-time  configurations.  Given  a  distributed  robotic  task  set  in  which 
sensor  and  actuator  tasks  have  temporal  and  locality  constraints  and  periods,  a  feasible 
schedule  is  derived  that  satisfies  all  constraints  and  guarantees  the  deadlines  of  each  task 
instance.  Prior  analysis  of  control  configurations  will  enable  the  uBot  team  to  consider  dis¬ 
tributing  computational  load  the  same  way  that  it  considered  distribting  motor  and  sensory 
tasks. 

The  scheduling  algorithm  has  been  implemented  and  the  effect  of  the  number  of  processors, 
the  task  size,  and  communication  costs  have  been  tested  by  the  simulation.  We  are  using 
the  algorithm  to  do  a  detailed  run-time  analysis  for  our  leader-follower  control  configuration. 
Results  provide  an  upper  bound  on  the  number  of  robots  that  may  be  bound  together  in  a 
coordinated  behavior  before  control  processes  are  no  longer  schdulable.  At  this  point,  team 
hierarchies  must  be  employed  to  coordinate  groups  of  robots. 

Wearable  Interfaces 

Our  implementation  is  based  on  a  Xybernaut  wearable  computer  as  a  computational  and 
display  device.  A  three-dimensional  graphics  engine  renders  virtual  objects  on  the  heads- 
up-display  at  approximately  10  frames-per-second.  Head  orientation  is  sensed  in  real  time 
using  an  accelerometer-corrected  gyroscope.  Current  head  position  is  provided  either  by  our 
smart  sensor  network  or  via  GPS.  Demonstrations  have  been  constructed  in  which  a  human 
collaborator  subscribes  to  the  sensor  stream  generated  by  a  distributed  robot  team  to: 
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•  see  through  visual  occluders; 

•  annotate  the  environment  with  auxiliary  cues  highlighting  hazards,  safe  paths,  and 
exits  in  a  search-and-rescue  scenario;  and 

•  illustrate  the  performance  and  progress  of  a  distributed  robot  search  task 

The  VR  interface  allows  the  user  to  walk  around  within  this  virtual  space  and  even  achieve 
birds-eye  views  of  the  state  of  the  environment. 


Accomplishments:  We  have  produced  deliverable  technologies  in  several  of  the  research 
foci. 

1.  A  parametric  motion  controller  that  has  been  used  to  implement  interesting  classes  of 
coordinated  behavior,  including: 

•  search  and  mapping  behavior, 

•  leader-follower  behavior, 

•  a  multi-robot  behavior  for  preserving  network  connectivity  among  peers  in  a  co¬ 
ordinated  task,  and 

•  a  bounded-overwatch  behavior  that  localizes  individuals  within  a  coordinated 
team  using  virtual  stereo 

2.  Demonstrated  control  expressions  that  compile  into  distributed  implementations  that 
satisfy  globed  specifications  and  demonstrated  how  equivalent  functional  configurations 
vary  with  respect  to  time,  energy,  and  comm  bandwidth, 

3.  Mechanisms  for  recovering  hidden  state  in  distributed  systems. 

4.  10  UMass  “uBots”  each  with  onboard  StrongArm  processors,  embedded  Scenix  con¬ 
trollers  for  IR  obstacle  detect,  inter-robot  communication,  and  RF  data  link.  Monoc¬ 
ular  color  vision  systems  have  been  prototyped  and  installed  on  some  of  the  robots  to 
transmits  video  to  an  off-board  a  vision  host  (a  933MHz  Pentium  III). 

5.  Policies  for  Group  Coordination:  line-of-sight  constraints  in  communicating  teams  of 
robots,  and  bounded  overwatch  strategies  for  preserving  a  global  frame. 

6.  Policies  for  exploration  and  map  making  while  maintaining  localization 

7.  Demonstrated  RT  analysis  and  performance  bounds  on  a  n-robot  search  configuration. 

8.  Virtual  environments  for  robot  control  and  situation  assessment  on  the  basis  of  percepts 
generated  during  a  distributed  multi-robot  search  task  at  10  frames  per  second. 
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9.  A  network-distributed  object  representation  system  that  distributes  three-dimensional, 
virtual  objects  across  a  set  of  servers  located  on  a  common  network. 

(a)  Visualization  of  the  UMass  Torso  robot  is  reflected  in  real  time  into  a  virtual 
environment. 

(b)  “X-ray  vision”  demonstration.  The  user  is  able  to  perceive  remote,  moving  objects 
that  are  occluded  by  walls  or  other  objects. 

(c)  Dynamic  display  of  distributed  robot  search  task:  robot  positions  are  displayed 
in  a  virtual  world;  as  the  robots  discover  walls  and  target  objects  in  this  space, 
these  objects  are  added  to  the  model.  The  VE  interface  allows  the  user  to  walk 
around  within  this  virtual  space  and  even  achieve  birds-eye  views  of  the  state  of 
the  environment. 

(d)  Mosaicing  visual  streams  into  a  texture-mapped  virtual  environment. 

10.  UMass  has  begun  an  implementation  of  a  grasping  control  architecture  for  the  NASA 
Robonaut  program  based  on  the  ideas  developed  under  the  SDR  and  MARS  programs. 

The  following  text  discusses  our  progress  in  more  detail  and  on  several  dimensions  of  the 

project. 

Hardware  -  UMass  UBots  The  UMass  SDR  project  has  produced  10  motion  platforms 
(UMASS  uBots)  that  comply  to  the  form  factor  guidelines  underlying  the  program. 
Coordinated  controllers  have  been  implemented  on  the  two  existing  platforms.  The 
UMASS  uBot  consists  of  multiple  layers:  a  vision  (and  future  acoustic)  sensor  layer, 
an  IR  layer  for  obstacle  detection,  future  capacity  for  an  IRDA  broadcast  communi¬ 
cation  channel,  a  DIMMPC  implementation  with  RT  kernel,  and  a  QNX  StrongArm 
version,  and  I2C  communications  between  embedded  Scenix  subsystems.  Currently, 
IR  and  motor  power  subsystems  are  running  with  stable  firmware.  In  addition,  an  RF 
rangeLAN  point-to-point  communication  facility  for  robots  to  communicate  with  their 
peers  has  been  implemented.  The  first  robot  demonstrations  of  the  control  structures 
proposed  are  running  in  the  laboratory  and  demonstrations  are  being  constructed  for 
a  series  of  multi-robot  search  tasks  in  a  model  office-style  interior  space. 

Software  -  Control  Polices  and  Representations  Intra-robot  communications  between 
multiple  embedded  controllers  is  complete  and  tested.  This  firmware  is  the  subject  of 
the  first  application  of  the  RT  analysis  software.  Among  the  first  results,  several 
distributed  leader-follower  control  configurations  have  been  guaranteed  for  a  bounded 
number  of  robots  (up  to  35  for  one  configuration).  RF  communications  between  robots 
through  a  logical  global  memory  model  is  also  complete  and  currently  under  testing. 
Applications  software  has  also  been  developed  for  a  variety  of  control  tasks  including: 
multi-robot  search  tasks,  bounded  overwatch,  leader-follower  applications,  and  swarm 
controllers.  In  addition,  laboratory  benchtop  vision  systems  for  our  platforms  have 
been  completed  and  data  sets  have  been  generated  with  which  to  learn  to  recognize 
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peers  in  a  partially  occluded  environment  and  to  extract  features  of  the  environment 
as  constraints  to  motion  control  tasks. 

Developmental  Programming  Paradigm  Examples  of  the  control  methodology  whereby 
simple  skill  sets  in  single  robots  precede  peer-to-peer  coordination  strategies,  and  fi¬ 
nally,  team-oriented  collaborations  have  been  constructed.  The  idea  is  to  compile 
control  knowledge  incrementally  in  a  developmental  sequence. 

Experiments  with  the  interaction-based  representation  for  motor  control  have  been 
demonstrated  in  simulation  that  estimate  control  state  by  observing  the  transient  re¬ 
sponse  of  a  discrete  set  of  linear  impedance  controllers.  Results  verify  the  generalization 
of  policies  based  on  these  representations  to  novel  run-time  contexts.  Policies  have  been 
constructed  for  optimal  energy,  time,  and  precision  controllers. 

To  structure  peer-to-peer  interactions,  a  control  constraint  between  coordinated  pairs 
of  robots  has  been  developed  called  the  “push”  and  “pull”  kinematic  conditions.  These 
conditions  specify  kinematic  properties  of  a  chain  of  interacting  robots  that  can  express 
leader-follower  relationships.  Moreover,  these  kinematic  conditions  have  been  used  to 
express  line-of-sight  connectivity  in  a  communication  network  with  an  arbitrarily  large 
number  of  robots. 

Stable  Real-Time  Adaptation  An  extension  to  vanilla  C-syntax  has  been  developed  with 
which  to  support  RT  analysis  in  concurrent,  embedded  applications.  Programs  written 
with  .be  code  extensions  contain  descriptions  of  periodic  the  control  tasks  that  specify 
the  robots  behavior  under  real-time  constraints.  Design  rules  are  employed  to  con¬ 
quer  the  complexity  of  RT  analysis  using  “divide  and  conquer”  techniques,  by  folding 
together  .be  program  phases  that  have  identical  time/resource  requirements,  and  by 
partitioning  .be  programs  into  adaptive  sets.  We  have  implemented  a  real-time  ker¬ 
nel  and  a  graphical  user  interface  for  developing  .be  programs  for  the  DIMMPC.  The 
GUI  serves  as  the  front  end  for  advanced  off-line  analysis  and  compilation  of  the  .be 
programs.  Once  the  RT  programs  have  been  specified  and  compiled,  the  system  guar¬ 
antees  that  they  will  meet  runtime  deadlines.  We  plan  to  conduct  a  complete  analysis 
and  verification  of  the  platform  firmware  during  summer  of  2001  and  subsequently,  to 
verify  higher-level  control  applications  as  well. 


Technology  Transition:  Twelve  students  are  involved  in  our  SDR  project. 

We  are  continuing  to  identify  opportunities  for  the  direct  transition  of  technology  under  the 
project  to  DOD  applications.  We  recently  acquired  a  small  NASA  contract  to  deliver  our 
controllers  for  applications  to  the  Robonaut  program. 

We  recently  ported  control  code  for  autonomous  grasping  and  manipulation  gaits  for  robot 
hands  to  our  colleagues  at  NASA-JSC  for  use  in  the  Robonaut  program  and  our  simulation 
for  multi-robot  search  controllers  that  maintain  line  of  sight  was  transferred  to  colleagues  at 
SPAWAR. 
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